An Enhanced Multimodal Sound Localization with Humanlike Auditory System for Intelligent Service Robots

This paper is concerned with an enhanced multimodal sound localization with humanlike auditory system for a network-based intelligent service robot, which exploits strong information technology infrastructure. The objective of this paper is to integrate several audiovisual-based Human-Robot Interaction (HRI) components that can naturally interact between human and robot through audiovisual information obtained from robot camera and microphones in the noisy environments or the presence of multiple persons. The proposed approach is comprised of two main stages. The first stage performs speech recognition, sound localization, and speaker recognition to know whether the user calls the robot or not as well as the direction and identification of the caller respectively, when someone calls robot’s name. In the second stage, an intelligent robot moves forward to the specific caller based on multiple face detection/recognition with the aid of the information identified by speaker recognition among multiple persons. The robot platform used in this study is WEVER, which is a network-based intelligent service robot developed in Electronics and Telecommunication Research Institute. This robot refers to an Ubiquitous Robotic Companion (URC) that provides necessary services anytime and anywhere. The effectiveness of the proposed approach is compared with other multimodal methods and sound localization itself.